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Because design plays a central role in engineering, it is important for engineering education 
programs to prepare students with design skills. 

Implications of Findings 

By describing both novice and expert approaches to engineering design, researchers are 
contributing to the formulation of more specific design learning outcomes that may be addressed 
in curriculum design and program planning. The narrative analysis of expert problem scoping 
behavior suggested a sophisticated approach to situating problems and solutions in context. 


The narrative analysis provided some promising directions for future research. It allowed us to 
identify significant themes in their narratives, potentially different from what the novices had 
tended to highlight - existing engineered solutions, alternative design solutions, costs and 
benefits, priorities, and history. 


More interestingly, narrative analysis allowed us to see 
the relationships between and among an experts' ideas 
and develop new understandings of what those 
relationships look like and how they reflect expert 
designers' thought processes. 


Even though they were not asked 
to do so, the experts began to 
consider alternative design 
solutions and wall materials. 


Method and Background 

In this study we used two methods for analyzing expert data: verbal protocol analysis (VP A) and 
narrative analysis. VPA has been effectively used to describe the design processes employed by 
engineering students, expert designers, and expert-novice comparative research. VPA involves 
asking participants to "think aloud" while performing the following design task: In the past, the 
Midwest has experienced massive flooding of the Mississippi River. What factors would you 
take into account in designing a retaining wall system for the Mississippi? Each participant 
worked individually and had up to 30 minutes to complete the design task. Verbal responses 
were audio recorded and transcribed for subsequent coding and interpretation. Narrative analysis 
entails a close reading of the transcripts seeking to identify content categories (particularly those 
not highlighted in the VPA) but also to trace the logic or structure underlying the respondent's 
answer. (See full length paper at the link below for a complete description of methods.) 

The expert data came from a subset of four responses purposefully selected from a pre-existing 
pool of 19 experienced engineers, representing six engineering specializations, identified by their 
peers at work as expert designers. 


One main objective was to choose the "most expert" participants. This was accomplished by 
considering years of professional experience, as well as responses to a "playground design" task, 
which they completed prior to the task discussed in this paper. The second objective was to 
choose participants to maximize diversity of perspectives and was accomplished by purposefully 
including one of the women among the four, and selected diverse participants based on their area 
of specialization. 

The concepts of design detail and design context enable quantification and comparison of the 
breadth of problem scoping among expert responses. Ideas focused on the wall or the water from 
a technical or logistical perspective were interpreted to be oriented toward the details of the 
design problem. All other ideas were considered to be oriented toward the context of the design 
problem. There was no common pattern among the four experts in terms of the numbers of 
segments, or the distribution of these segments across the coding scheme. 

The authors then situated this expert data in novice research from the Academic Pathways Study 
research element of the NSF-funded Center the Advancement of Engineering Education. A direct 
comparison of the expert and novice samples cannot be made because the methods for collecting 
data resulted in vastly different numbers of segments. APS data were collected from students at 
four of the CAEE partner institutions using surveys, structured and semi-structured interviews, 
ethnographic observations, and simple engineering tasks. During the first year of the study, the 
MWE task was administered to 124 freshmen. Instead of thinking aloud, students were given ten 
minutes to write down their answers. The transcripts were segmented and coded the same way as 
described for the expert data above. 

The data for each expert participant consisted of a transcribed response to the verbal protocol on 
the Midwest floods (MWE) design task. Each transcript was segmented into distinct "though 
units," or ideas. A coding scheme was applied in order to systematically measure and categorize 
design processes. Segmented transcripts were coded on two dimensions of problem scoping 
breadth: physical location (wall, water, riverbank, or the wider surroundings beyond) and frame 
of reference (technical, logistical, natural, or social). 

What We Found 

None of the experts in the sample used the full 30 minutes they were given to think aloud. The 
number of coded segments among the four experts ranges from 20 to 118, with a total of 228 
segments. The total number of segments includes the unique ideas a study participant offered in 
response to the MWE problem, as well as any "non-applicable" or “equivalent” segments. 

Novices’ ideas were distributed across physical locations in a similar fashion to those of the 
experts. As with the experts, about half of the novices’ segments were focused on the wall, twice 
as many as those focused on the water. As with the experts, the frame of reference codes are 
more evenly distributed across the four categories, but the distribution is somewhat different with 
novices considering more factors with a natural frame of reference than experts. 

In the narrative analysis of expert data, five kinds of factors stood out, either because they hadn’t 
seemed as pronounced in the novice data or because they emerged rather strongly in the expert 
data: 
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1 . The experts suggested looking at existing retaining wall systems and other existing 
engineered solutions for flooding. 

2. Even though they were not asked to do so, the experts began to eonsider alternative 
design solutions and wall materials. 

3. The experts weighed eosts and benefits throughout their narratives. 

4. The experts considered priorities based on costs and benefits. 

5. The experts situated their narratives in history. Each narrative had temporality: a past, a 
present, and in some cases, a future. 

The following observations were also made about the experts’ thinking processes: 

• One approach to the MWE was to state a broad factor, and then elaborate and expand 
upon it by brainstorming a list of related detailed factors. 

• Expert verbal responses often included logical connections among sets of related factors. 

• One method of organizing thoughts was to embed a set of related ideas within another set 
of related ideas. 

Eurther analyzing the transcripts for these four experts and including those of the other experts in 
the study should allow us to identify additional ways of thinking about and doing engineering 
and affirm or elaborate on the narrative theories presented here. Illuminating the though 
processes of expert designers can provide direction for improving the way we educate novices. 
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